Feat/interaction logs - #16859

…r scoping

Replace the rag_api/pgvector vector store with an AWS Bedrock Knowledge Base
(ministack-provisioned locally): files are written to the KB's S3 data source
with a .metadata.json sidecar, ingestion jobs are debounced, and file_search
and attached-file context retrieve via bedrock-agent-runtime Retrieve.

User separation: retrieveForFiles scopes a user's own uploads by file_id and
user_id, agent files by file_id alone, re-checks every result against the
entitled set, and returns nothing without entitled files. Unscoped retrieves
are refused, and object-key filenames are sanitised.

Also unify the ragConfigured checks behind isBedrockKbConfigured(), blank
RAG_API_URL for the app so text extraction uses the native parser, and
rewrite the stale rag_api fileSearch tests plus add scope specs.

This reverts commit d091d5d.

This reverts commit def82c0.

This reverts commit 0b2e49a.

… per-user scoping"

This reverts commit 9260524.

The revert left RAG_API_URL de-indented to the top of `services`, so the
override declared it as a service instead of passing it to the API, and
rag_api/vectordb stayed parked behind the `legacy-rag` profile even though
the API depends on rag_api again.

Put RAG_API_URL back in the API environment and drop the profile gates so
both services start with the stack.

DocumentDB requires TLS verified against the AWS CA bundle, which Mongoose
only picks up through `tlsCAFile`. Pass MONGO_TLS_CA_FILE through to the
connection options when it is set; local MongoDB leaves it empty and
connects as before.

In the compose override the API now takes MONGO_URI from .env instead of a
blank value, forwards MONGO_TLS_CA_FILE, and mounts ./certs/documentdb
read-only for the bundle (the bundle itself is gitignored).

The local mongodb service becomes an opt-in single-node replica set behind
the `mongodb-compat` profile, so the compatibility suites can exercise
transactions. It is a baseline only, not a DocumentDB emulator.

Cover the network path (VPN or tunnel), the CA bundle, the MONGO_URI
parameters each connection type needs, and the `mongodb-compat` baseline
profile.

Real Bedrock inference needs real credentials, while the rest of the local
stack keeps talking to ministack with static test keys. Mount the host's
AWS config directory (AWS_PROFILE_DIR) read-only into the API container and
pass BEDROCK_AWS_PROFILE and BEDROCK_AWS_DEFAULT_REGION through.

The global AWS_ENDPOINT_URL points every SDK client at ministack, including
the ones that resolve a profile. Pin the SSO, SSO OIDC and STS endpoints to
real AWS so profile and SSO resolution still work.

Both directories are runtime volumes. Without the exclusion `COPY . .` sent
several gigabytes of model blobs to the daemon on every build and baked
their private keys into the image.

Offer Amazon Nova Lite v1 alongside Claude Sonnet 5. eu-west-2 serves it
on-demand only, so it is listed by its foundation-model ID rather than an
`eu.` inference profile.

Adds an `analytics` database of read-only views over the HyperDX and
Langfuse tables: UI actions and page views, LLM traces, model calls
(model, tokens, cost, latency) and feedback scores, plus a `timeline`
view that puts all of them on one per-user timeline in the column layout
HyperDX expects of a log source. The views leave out every prompt,
response, metadata field and feedback comment.

HyperDX's ClickHouse user may now SELECT from these views but still
cannot read Langfuse's tables. A one-shot clickhouse-analytics-init
Compose service creates the views once Langfuse and the collector have
created their tables. The SQL drops and re-creates the views instead of
using CREATE OR REPLACE, which fails on the EFS volume ClickHouse uses
on AWS.

Adds a "Usage timeline" HyperDX source over the analytics.timeline view,
a saved search of the same name, and an "LLM usage" dashboard: model
calls, tokens, cost, response time, feedback and failed calls over time,
usage per user, UI actions and the event timeline, with User, Model and
Event source filters.

HyperDX applies DEFAULT_SOURCES only to a team that has no sources yet,
and its own dashboard provisioner needs per-installation source ids, so
the files name their source and connection instead. provision.js runs on
the HyperDX image, resolves those names per team with HyperDX's own
models and validation, and upserts everything by name. Locally it runs
as the one-shot hyperdx-provision Compose service.

Langfuse's LANGFUSE_INIT_* bootstrap only creates the admin user, so a
rotated LANGFUSE_INIT_USER_PASSWORD never reached an existing admin and
locked them out. admin.js runs on the Langfuse image as a one-off task
and sets the admin's password from the environment, hashing it with the
same bcryptjs library and cost Langfuse uses. It writes only when the
password differs and changes nothing before the admin exists.

OpenTofu hashes librechat.*.yaml and the files under otel/ into image
tags and one-off task triggers. A CRLF checkout on one machine would
change those hashes, so every apply from it would rebuild images and
re-run tasks that an apply from another machine had just settled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The OpenTofu stack moved to terraform/ at the repo root, and its files
were regrouped by service: the analytics views now live in clickhouse.tf.
Comments only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A HyperDX source links to others by id (traceSourceId, logSourceId,
metricSourceId, sessionSourceId), and ids differ per installation. The
provisioning files now name the linked source instead, as they already
do for the connection. provision.js writes every source without its
links first, then again with each linked name resolved to that team's
id, so sources may link to each other in either direction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Researchers had to switch to Langfuse for anything beyond counts: the
analytics views left out conversation content, so HyperDX showed that a
user rated a reply but not why, nor what was asked or answered. The
views now carry it, so HyperDX covers what Langfuse records.

- analytics.spans: every span from Langfuse's events_full (events_core
  truncates input and output to 200 characters), with full input and
  output, status, time to first token, and model settings, usage, cost,
  tool calls, tool definitions, tags and metadata as attributes. A new
  "Conversation traces" HyperDX source reads it in the trace layout, so
  a turn opens as a tree; the usage timeline links to it by TraceId.
- analytics.chat_turns: each turn's message and reply, as `chat` rows on
  the timeline with the message as Body, so search finds it.
- analytics.feedback: the reason (metadata['tag']), the comment with any
  text the user typed, and the score's metadata.
- The timeline gains Reason, Comment, Input, Output, StatusMessage and
  TimeToFirstTokenMs; the LLM usage dashboard gains feedback by reason,
  feedback comments, conversations, tool calls, agent steps, errors and
  time to first token.

This reverses the views' earlier content-free design: anyone who can
sign in to HyperDX can now read what users wrote, the replies, tool
inputs and outputs, and feedback comments. HyperDX still reads only the
views, not Langfuse's tables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>